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ABSTRACT 

Th€ puipoaes of tha paper vere. to develop a structure 
by which asaessfflent modela oould be uniguely chacactici^ea aci to 
develop assasaMnt aodcls and their asaooiatea cooputei: pEogxaia^ 
which coald hm used for pocsonael research purposes* To these ends 
two aspects of assessnant^ neasurettent and evaltiatian^ ««3:e 
identified- rcheir distinc tioii was ffiainly in the prcciss u^afl to 
respond to the ItaBs of an asseismant instjruinant* Tha pateie*ar^ of 
the as^esamexit models were identified* These were Trarlausay coBtolned 
and opeEatioDalized into either neasurenaiit or evaluation lodels* Ihe 
intention of this process uas to make evaluation ©ore 
aeasiireaeiit-Uke by using and atveloping evaluation ffloaals, ior 
polychctoiDoa^ reaponsa inforDation^ iihich were patterned aft« Ba^ch 
latent-^trait measaremeiit nodels, also deFeloped naxe scifle sp«c3.al 
models for use in decision Making paradigms and for mmmmmmt 
situations for which aii estimation sample does oc does no* eais** 
Analysis of these ffiodals and thtir associatedl data «te pxovdded by a 
system of conpater prograDs callea PEXBl. In all, five prcgtana were, 
developed, illustrated^ ana doaumented. applicatiois oi tlis^ models 
and computer programs to persoanel reeiaroh areas o* tbi mvaf 
Personnel Eeaearch and Davelopntnt Center research pregran 
discussed* (IC) 
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Tilt pisTp^ae q£ ttiis report is to dev^Rlop the framework for a 
s tr^jctmjre c? f asssssmfinC" and to pro\fide procedures for tha anal^psis 

0 f " 'as^i^Sifi€Jtt'' ffodiL^ dey^aleped frORi this stTUCture^ The author 
finals 4ha:t this enaia^or ts xiecejsaiey beoausa of the apparent eoiifuiloti 
t3iat tlte t^jlc of asseestnent appears to have on the ntnds of many 
ImGlvm^ Im personnil research activities, We shall proceed by cxamiaing 
elii fuffida meaita 1 ^sptcts of a^s^ssment procedfureSi d eve loping asaesainent 
tnedela ^ and de^ejoping coffiputet programs for the analyils of these models, 

Ttie ^oplc 00 tliis rep^tt tg asseseniimt for peraonnel projaccs* 
TBe auchaT assajniei th^t th^ reader Is in agreement vlth hint that the 
fmnieworlc oS a ^seisment of peraomel R6d3 In a frightful mess and needs 
caaE^ifica^lons c^f. Andeti^n'a (19 73) Anatomy of Evaluation ^ which contatiis 

1 ve-ry la^ge lisc of artlcJeaj coacept^j definttlDna and techniquas relating 
to one asptct of as^easrtitri^> the page (1975) t It is Msumed chat Oim 
iutlxor and reader a^e in a j^een^cit that the topic needa ilmpllf icatlon 

icid clarirficitiorx with r^speet to the specification and definltlori of its 
ftjniJ^mental aspects ^ and tPat nim. practice of personnel agseasment can be 
IfCiproved fcy pDStLLlatittg in ilmple structural framet^ork and by the develop- 
ment aod mm o£ formaL aaiessmeiit En^dels, 

Tl^i author ftirth^r as^toed that the reader haa had a professionil 
eduMtLon \^Lth ip*«e trainlrig on measuremint and ataclitieal toples and Ig 
me^ti^atiid tov/acrd impr&vad pirso^ael RfiJ) and ^iahes to leam of a new 
actetiipt to chatacteffise a #inapL« and maanlngful etTucture for the topie 
urmde^ d isauffitcsn^ and^ at che i^nrii time, to learn of lome tt&^ assessment 
ra^d&ls and phitt associated compLiter programs whicli can be used on personnel 

Ty-pes o f ^Bg esgnient 

A*, Asp^cti of ^gg eesm^nt 

, B)r thse te^ ''personnel asaesimint" the author refers to the -'meisure^ 
menit" o3f e\rail\wtlon" ©f p^rsontitl^ The discliiction hBtvmm these pro- 
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cesses will ultiniateLy be realized by a technical description. We 
are assuraing alio that the ^'object*' being asiessed U usually a person, 
since vm are dealing pTimartly with parions In pereontiel research work. 
Our proceclures are^ upon occasion, alio applicable for assessing other 
types of objacts or entitlei such ae syetems, organlMticris , operatioini, 
programs, taBkm, etc^, but wa will not eniphasige these applicattDiis. 
Our purpDse primarily being to assess patsomel, 

T'he substructures J assesamerit and evaluation, each has an enorntoui 
literature ^ not to be reviewed here. The subtoplc "measurement" lias 
uhdergone a long and steady evolution^ eliminating in vartoua latent 
trait mad els and cOTiJoint nieafluremint procedures. The subtoplc -'eval- 
uation'' is more diffuse and lees mathematlGally structured, and has a 
literature in the education and psychology fields prtmarily rilatid to 
€\7aluaCions of-^offlpleK programs rather than personnel. These a valuations 
frequently consiittng of meaaureTttenta or their functions, We shall not 
pursue this course in our development ^ rather ^e shall tend to make the 
process of personnfcl evaluation have analysis procedures siinllar to 
those of measuremeiit . 

Elements of j\ssessment 

We have already identified the object or entit y of ass essment as 
one of the elements of this topic* Another element is the asgessMiit 
instrument consisting of asseesnient items ^ each of which is ssaocieted 
%iith a set of item response categories . The asseismant insttfumeat in 
measureweht situations is a test or eKatninatlon whereas the instruTCnt 
in evaluaclon'situatlons is usually either a questionnaire, oheckllet or^ 
inventory. The reapcnae categories of the items of a teat may be ''tnulttple 
choice*" Por our puicposes we constder that each item has only two 
operational responsas categories, i*e,, correct or incorfect. Jvaluatloii 
instTuniants usually also have niultiple category reaponsas, vhi^h we aisume 
are also expressed as Levels of quality^ sevaral rather than two* 

Wa poaculata the eKistence of another asseastttent entity to liphlch we 
append the label "assessor," whose function it is to assess the objeat of 
aasessment, who is o^ isn't another person. In measureiQant the paraoii la 
using hia mm latent abilities or competencies to designate Che response 
categories of the items of the assessment instrunient* Whereas iot evalua- 
tion situationa, the asaesior usually utUlgea his sub^aetivs liidgnent 
about himself or another entity to designate the appropciati raspowi Mte* 
gortee of the iteflis of the assessment initrumant* 

As promised the previous two paragraphs provide our dtstln^tlon 
betwaen the concepts of measurement and evaluation* Wi suimnari^e tha 
various elements of assessment as followi in EKhlbit 1, Vm raaltitj in 
truth, that our diatlnction between '-meaiurement" and "evaluation-' is not 
aa sharp as we would like to nake It. More reallstlMlly we may consider 
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"Assessmenc" to be a bipolar concept with some assesstnenC situations 
more meaBurement-1 ike than ^vaLustloa-llka or vice verso. Fortunately 
Che dtstlnction Is not critical and an appropriate asaessment model for 
some R&D assesBmant project can ea^idly be located flmong those we shall 
propose • 

In additLon to these factors^ wc consider some further elements of 
importance* One of these is called the Assessme nt 5atiiple> which consists 
of those entitles to be assessed once the assessment procedure has been 
formalized after an analysis, CTits analysts la usuQlly made on data 
provided by an Estimation Sample gtom vhlch the pafarneters of the assessment 
model are cstimaCed and the" scoring procedure forrnaliged^ The purpose 
of the analysis is to develop Che scoring procedure to be used for 
providing quantified magnitudes estimating abtllttcs or competencies Ln 
measurenidnt situations or the 'Values'' of the asseB.^ees in evaluation 
situations. Other purposea are to analyse the assessment instrument items 
and to examine the -'fit" of the response data to UiVg assessnient modeU 

C* Asses_snient Model Spectrum 

In order to characterize the saseBSment systeni proposed herein^ '^e 
wish to define a set of symbols and their deflnitloris which collectively 
we designate as the Aasessinent Model Spectruni, The purpose of this spectrum 
is to provide a simple structural characterization of an asseBSmetit 
problem and provide for the use o£ this spectrum foir Ideiitlfying the 
approprlato assessment model analysis. Our spectrum conslits of the 
foilowlng nine parameters each having one or more corresponding designators 
as shown In EKhlblt 2, • 

The first paramoter, Aspect, Indicates that the assessor is considering 
the assessment problem to be either a meeeurement probleni or an evaluatlDn 
problem in the sense described previously* Our assessment procedure 
assumes that the assessor Is assessing basic characterlitlcs or traits 
that are not directly observable and therefDre these traits are referred to 
as latent traits g or he is evaluating soina value. It is the purpose of the 
process to estimate a person's standing on a value or trait, which can In 
turn be used for other operatioaal or research purposes* In measurenient 
situations I these traits are ref erred to as abilities, v^hereas for 
evaluation sttuatlons^ they are refarred to as values, in each case being 
further characterized by the name of the abtllty or value being assessed. 
This name betng listed as the second assessment spectrum parameter. 

The parameter "Initrument" refers to the type of assesiment instrument 
utlllEedy most frequent types feeing testsi questloniialrea, or checkiists. 
The number of Items on the assessnieftt Itistrument Is listed in the fifth 
parameter. Our procedures requlra the assessnient Imtruinent, no matter 
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EXHIBIT 2 

Assessment Model Spectrum Parameters and Subpa rameters 



Parameter Hanm Subparanieter 

M ^ measurejiient (primarily) 
^^P®^^ evaluation (priniarlly) 



Tra 1 1 



L ^ latent trait name to be measured 
V - value evaluQlcd 



P — person 
EntitY S - system 



Inst ruinent 



0 - abject or uglier 

T - test or exsrni nat ion 
il ^ ques t Id nna i re 
D ^ checkl i st 
0 ^ othe r 



Items I - number of assessment instrument i tenis 

Responses M - nuraber cf response categories 

llcation Sample N = size of application sample 

Estimation Sample E » size of es t i mat I on sample 

A = Intttrit flbllitY caiiip!3t«3ncy 

Assessment ProcBas ^ ^ subjective judgment 
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lAat Its fotm, to consist of a subset of statemenns, questions, ot whate\r«, 
which may be labelad as items. Each item is eesoclated with a set of 
QOmon raspoMe categories, the number of \^ich m describe by the p^raroeter 
M» If M ^ 2| the response categoriei are described m binary or dtrtotonioui , 
^ere If M > 2^ the response category type ii called ^olychptomoua* tn 
imlttple choica tests, M ^ 2, stnta the response is judged correct or 
incorrect. In spite of the fact that the ttstee has more than two ^Ueraativras 
from vhich to choose tot each Item, Our concern is fcr the numbef of 
•*Dparatianir- response categories to be used for aMlysts purposei iiid 
not for the number of Instrumental response catagortea* In our aaseisment 
model syatfinip the response categories for evaluatloti instruments represent 
levels of quality possibly reallEed by the essessee relative to the items' 
substantive content^ For eKample, for the item "How wuld you rat^ this 
initruQtor v;ith respect to his general teacKlng ability?", for yhtch we 
may have the responae categories- 

poor 
fair 

adequate 
good J or 
excellent 

Finally^ we are concerned with the existence and size of two m^tm 
af data resulting from persons for Tihom reaponie information may be 
available. The first sample, called the Appllcatlori sample is not 
ordinarily used for analysis. It is this sample ^ the members of ^ich 
are to be aasessedj once the assessment procedure is defined. In Emm 
cases, this application sample is not Immediately available, since the 
assesssent procedure is being developed and to be applied some time In 
the future. The other sample^ called the Eitlmatlori sample. If it 
^IstSj provides the response data to be utilised for it estimating the 
parameters of the assessment model. Part of our aisessment system provldea 
aodels to be utilised for those occasions^ niually ocmrrtng in evaluation 
situations, for which an EstlMtlon amnple does or cannot exist because of 
©peratlDnal constraints* There are occasions, usually in measurement 
situations, when the Application and EstlMtion samplis are Identical 
This olrcumstance causes no difficulty for the models of our systetti* 
For most measurement model analysis, we require an Eatimation sample of 
adequate sise. 

The parameter "entity" is either a person, syitMi^ object or some 
entity which can be observed producing or made to produce information 
which can be transfDrmed Into asseisMnt instrument response categories* 

Models of Assessment 

The modern scientific approach to meaaureraent theory my be satd to 
have begun with the publication of Lord's Cl'952) "A Theory of Test Scores'' 
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K ^ T + E ^here 



K repreaents the observed test gcore, T repreitnts the ttue scores 
and E represents random arror. Lord extensively published research work 
on this and related true score models during the score of yeati 1950-1970. 

A related modelj ^ich for lack oi a better name, we eh^ll call the 
■•Minnesota" model is 

(2) * R ^ A + I + e whmre 

R represents the response made to one of ttie Items of a test* 
A represents the ability of the item respondent j I represents the item 
effect^ and e ts the random error aisociated with the Itern response* In 
our opinion^ (2) representa the metiurement model commonly assumed to 
eKist in the minds of many personnel research workeicSj althoijgh its analysis 
is not carried forth in the popular tTieasurenient te?^t bookie Including 
Gullikaen's (1950)5 In the fields of education and psychology* Prac tit lofteri 
Lmtending to utilize the concept of <2) usually proceed to conitruct a 
measuring device whose item reiponses are then judged as correct or Incorrect 
and which are accumulated into total scores* These scores ate not for the 
purpose of estimating the ability of the testee as would be the objective if 
the practitioner were interested in measureinent p but the scores are 
frequently utilized for relating them to other Informiation and as euch 
represent statisticals rather than tneaaurenient , objectiv^ea* A.fl a conBequenci 
^e attach the terms ■'pseudo-measureinent*' or quasi-meisurenient" to this 
practice # 

m 

Various classes of la tent* trait faodele are described by Lord and 
Movik (1968)* Distinctions of these mLodele are accomplished by denoting 
the parameters involved in the model specification* These parameters may 
be listed in Exhibit 3, 

Some or all of these parameters are asiembled Into a mathamattcal 
eKpresslon describing the probability of a given type of rCnjl) as a 
function of the parameters used in the model* This model is then said to 
be a latent trait measurement model* Ihe so-called Birnbaum C1968) 
three-item parameter model utilizes the three-item paramitersS <i) * d(l), 
and g(i) as well as a (n) in develDptng an e^pressioti for the probability 
of a correct response* The Birnbauin (1968) two^ttem paranieter model 
involves the two^ttem parameters ot& (i) and d (i) ^ m well as a <n) , for the 
same purpose* 



^he label "Minnesota" Is approprtate since ProfMsora Palifter 0* 
johnson* Cyril J. Hoyt, and others used thts model, during 1940-1950, 
as an analysis of variance model to Instruct studenti and carry out, 
and develop varioui psychometric analyses at the Unlverslcy of Minnesota. 
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EXHIBIT 3 
Parametars of Assessmeot Models 



a (a) I a real mmhmt repreaanclng the ability ©r 
latMt trait of the n^h perios. 



S (i)s a real nimbmr rapreientlng the "difficulty' 



Of tha 1^^ itra 



d (1)^ a rwl number repwsenting the 1^^ ttea^a 
discrimination power. 



r (a, 1)^ the response made to the 1^^ ittm by the 



g (i): a real number representing the effact of 

luessiflg the correct answM to the l^n Item. 
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Rasch (I960) alio developed a variety of latent trait models 
o£ lAtch, the Binary Meaiurement Model, 3m, develops the expression for 
the probability of a correct reaponse as a function of 5 (i) and a (n) 
OR,iy* Ht ^as able to show tliat certain desired measuring propertlJi^ , 
called ^'objectivity'' resulted from this specification* Briefly^ obj®ctivlKy> 
m^ani that y& can achieve both 

<a) persoTi-free item measurement^ and 

(b) item-free person measurement by this spe cif ication* 

Th,^ Rasch model does not actually utili^^e 6 (i) t but rather, anothM 
p^irametar deeignated as e(i) which we may assume is conceptually mvA 
inversely related to 6 (i) and referred to as ^'item easiness, " 

In our assessment system, we shall utilise the BWi as the b$miB tot 
mMsuremant and shall utilise generalisations of this model for poly* 
chiocomous raaponses as the basis for ™klng quantitative evaluation * 
Mm do this to rnalntain a degree of continuity of process, and for the 
reason that the Rasch models represent the simplist latent trait madil 
an<l therefore can be analysed mathematically and computationally i^ith 
Enltiimal skill and effort. 

We conclude this section by Indieating the assumptions requi^rid by 
the BhW* These are: 

(1) lest items are scored dichotomously* 

(2) The probability of a correct response la only a function pX 
the ability, a (n) , of the respondent arid the easlne;^i^ £ (i), 
of the item* 

(3) Responses to the items are stochastically tndependem^ glv/eti 
OL (n) and e (1) , 

The prime System 

Introduction to PRI>ffi 

Any aasessment is the result of measuring or judging something* 
That iSi applying some assessment process or procedure ^ some nurnb^tr or 
code Is recorded as a result. The purpose of making an assessment is 
to use it for doing something* Hence the object of tnaking assessmtnti 
ii to provide a basis for action. 

If the assessment Is related to mental measurement, the *'isgeaimemt 
process-' le usually an intellectual function, such as problem iol^ilng 
or Che use of some memory process. In these kinds., of situational th^ 
"aisesstnent process'' is carried out by the person (entity) being miiiSurad« 

If the asaesament Is related to evaluation, the "assessment opigrittom" 
(a jud^ent) is often carried out by an evaluator with respect to the 
^rttity being evaluated* 
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In cither event, thfe nuttib^r or code resultlnj?, itm aaseBsmant is 
a spectftcation of a rePTSOnse (¥«itep,ory, i.e., a mviltipU choice selection 
or designation of a valm cac«Rory of aasesatnant Iftetmrnene Itama. 

As the acronym "PRM" indicates, tt is a aysmm 4woted to 
"instrumentalities" foe Tnaklng weasuren^nts and evalwatlons in peraonnel 
research problems. Thefle instwrnentalitles repreBBnt measurement/ 
evaluation models and tiaait a^aaclated computer ptogwi^i designed to 
operate on cofflputera availflblw to NPRDC personnel i 

The PRIME programi 0m inuffierated as follows s 

FRMj The Frequency R«tlo Method (Binary Measwement Model) 

TIMs The inapectioia) Model 

pAMs The PolyehotwnoW Aasessnient Model 

P1#I: The PolyChotomoUB Measurement Model 

HELP : The Handy ^ Empiiyical Uttle Priorlfclgey 

(HAXKO) : Auxiliary WlW Program 

The objective of the PRtWE system is to brlns "order" Into the topic 

of assessmont and to ptwide wdels, and their a89»ci#ttd computer programs, 

to workers In the field of laraonnel Research. 

We next provide a htimi w«rview of each in&trume'ntaliCy and ulti- 
mately indicate an appllcatiOrt, £nr it in Personnel Eesearch Programs, 

B. Briefs of nim ,l^ff ^B^m>snc jll,tj.es . 

FRM: The Frequenc y Rattth Method . This pragrttm ia related to the 
Rasch^Blnary Measuremetit^oWt"<BtM) discussed by Wooftan (1969, 1974) 
and represents a new and slmpl«t method for estimating the "easiness" 
of measuring instrument ieem». The program Is oHm^i as a poislble 
substitute for a more es^fcanilv* IBM 360/65 prograttt called "Max" In the 
NPRDC Computer Program WbMvy. FRM operates on tlie mwC GA 1830 com- 
puter system, Document^tloti an example using FRM for analyzing B^D^ 
data li provided In Mowan Covher (1975a). 

FRM is useful In ch»ae m^aauremant situations wh«re the BI#I applies 
and some cognitive don^in ch««»ctertstics, such as Intelligence, ability, 
aptitude or competency 4a required to be measured bjir items whose 
cesponses are judged opwatiowlly to be dichotowoua ^e.g., correct or 
incorrect) . B^M and FRM are applicable to most peraoanaX meaBurenvent 
problems where the ite.m scovfing paremeters are 0, 1% 

FMM* The Polychoewous Measur ement Model , Situations occur in 
Personnel Research, ho««ver, when it is not conv«nieri^ or desirable to 
cneasure an entity propfiety by using dlchotomously sewed items. ^ Such 
occasions arise when mi»«urto| attitudes, interests, i*nd motivations. 
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For theS6 'puriioses we can use a generalisation of the Raach BW\ to 
the polychotomous Item response case Rasch (1970). We refer to thle 
model aii WM^ Anderson <1973) developed a condttional maximum likeli- 
hood cofnputair program for this model. The author has attempted to secure 
this progrira Bor pRIlffi in the following ways i 

(a) mtd punching a listing of the program as contained in 
Andersen, et al* (1973); 

(h) itequesting a program copy from ETS*s Data Processing Division; 

(c) xrequest a program copy fromAllerupi F# and Sorberj G, (1975)* 

Procadur^ (a), because of poor printings produced a program deck 
that would not compile on NPRDC computers* It turned out, that in spite 
of Anderaet^ efc al, (1973) , ITS did not have an operating program either. 
They offared information about a similar proprietary University of Chicago 
program, l^ava requested a F^W copy from Denmark , and have confidence 
in receiving It but have ^ as yet^ not acquired it because of the delay due 
to distaiica** Keverthelesa we include Pjm in the PRIME System. The 
chief advintiige of Andersen's F^§l program is that it obtains maximum 
likelihood t^fclMtes of the PMi parameters including the scoring parameters 
of the polyahc^tomous responses to the asiessment tnstrument items* 

Wti la useful for measuring interests and attitudes as well as for 
analysing iv^luattve instrument datas 

PAMs Th^ Pol ychotomous Assessment Model* Because of early concern 
over acquiring Andersen's FhW program^ it was decided to develop an 
approximation programj called PAM, to the PMM program* This model handled 
polychotomou^ response data, but raated response categories Independently* 
Rather than ^atlmate the item scoring parameters, it defined them mathe- 
matieally^ The item paranieter estimation procedure used was that derived 
for use ^ith program FRM. Using the scoring parameters , entity scores 
can be dariv^4 and astimatea of entity attitude, Interest or whatever can 
be obtaitied^ 

Doeumant^tlon for PAH is cantalned in Moonan and Covher (1975b) and pro 
videa an aici»m|^l€ using SVIB data collected from naval personnel* 

TIM: Tht Inspection Model * Occailonally in Personnel Reiearch, 
there occura H situation wherein it is either Infeasible or impossible 
to collect r^aponse information on the items of an assessment Instrument 
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from an EatlMtion sample i ymt issesiments tnuit be made on in 
Asseasmant sample* If the aiieseffltn.ta are evaluatiofii and thm itmm 
are polychotameus, the tiM modal is appraprlate^ T^h^ l teni pacine tec 
information Is Dbtained frora Ittni "tmpoirtancai'' aupplis^ by the 
evaiuators 

Entity "scpres" are obttimd from a wight^d avetagi Chi fr^quan- 
clea of Item raspsnse categDrLas (obtainad duritig evaluation) mi icim 
rasponae acorlng parameters, A TIM modal entity parainetar ^ asiociatsa 
with each pesilble seore is provided by the program. S«eh in istiniate 
provldas knwladge of the amount of the qtjallEy^ measurid fcy Chi liaessmsiit 
instrufnent J poasessed by the avaluatad antity, PleaSi note that IIH 
the first PRIlffi Instrumentality discussad which does n&t r^quira an 
Estimation ample vhich provides reeponsa data from ^hl eh item p^rtmatcrs 
can be eititnatad. Documentation fo^ TIM is con«tna«i Ifl Mconan» it al • 
(1975c). 

HELP; Handy Empirical. Little Pglarltizer^ Iti tlios« aSiepssmant 
altuatlons frequently" asaoelatad with dacliien itateing, and others, ^ne 
is obliged to prov^ide quantified raeaiures of values or titllit3^< Ofceti 
this process iiivolvas tha eonslderation of seyaral factors of impsrCatice* 
HELP calls these factors "Dtmeiislona of Value" and requires thi Lmsa^ t« 
provide a consistent ratio scale of the importasica (to hlttt^ o4 e^eh 
Dimension. Each entity of the Assessment sanipla Is than jnidged tfa €o 
qualtty on each Dlmeniton, This information Is then lynthisized by the 
HELP model into a quanttfled measure of entity value* These wlaas 
Chen being used yith other decision makingp gtatlettca or Opemttona 
Research models for the purposes at hand* 

Such procedures, whleh quantif 3? multiple-dimensional judsneffts^ 
are rarely available and few are as ilmple mm HELPt Ceini^quemitly it 18 
very important for Personnel Research purposei. The HELP docminientatioii 
Is given in Moonan and Covher (197Sd), 

MAKKO* The purpose of this prcgram Is to provide fiiaj^itnuBi likell-^ 
hood estimation of the "value" parameter in the TIM and P^H mcdel pifograifiSi 
MMO is the polyehotoraous generalisation of eubroutine M^MO ua^d in 
FTOf for the dichotomous response case* The program prints a ^^BQC^tlm 
table" associating a scor" iriv^ed from a polychotomou^ a^sasMicit dne-trw 
ment with the esttnwte of thi value paramater of tha BrnmrnimM ffiod^U 
The method of estiMtton is that of iteration on the modeL iq^atioii on 
p* 2 of Moon and Covher (1975e). 

We have now listed and briefly desorlbe<l min PRM progr^nia - 

For clarification purpoias the follwing charts E^<hit»it 4, mni 
Isdiiblt 5 s characterise the relationahip of 4 of the pRlhE tnstrumetitiili^ 
tias in Exhibit 4, and the aseissme^t parameters of lach cQOd#la«i ihown 
in Exhibit 5. 
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EXHIBIT 4 

^tXationa of Four PRIJffi Instrumeatalleies 



Operational 


Item lesponee Data 


Ob J active Sub i ect ive 
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— -..^ — -. — — . — — 
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HEIP 
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1,M 
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E 
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L 
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P 


S.S.O 
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Q.C 


Q.C 




Q.C 


Reaponsei 




Vt>2 


fez 


^2 




EscimAtioa 


■yes 


No 


Yea 


Yea 


m 


^plt Cation 


Tfaa 


ITes 


Yea 


Yea 


Yes 



Hcta . Sae tohlblt 2 for dtftnltions of 
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V. PRIME Applleations and "Miasurement lads'' 



BicauSi the pursuit of g^d TOeasuramant and evaluation is mn acttvt 
om in the course of Persotinel Research, we ihould expect that tht 
PRIME Itiitrymintallttes ShQ^ld enjoy wide utility. 

In the miaawement timU there exists the tendency toward "fadSi" 
Me have already aeen e long period of the fad of "Norm-feferenced 
measureraiiit*^ whetein test Items and teats are ealibtated relatlvi m a 
aampie cf persotii from poptilatlons of interest* 

Wi are presently %?ttti€Siing the beglniiing of the pitLod ©£ chi fad 
of *'Crt tar ion-Re ferenced meaaurement" i^hirein item and teit aalibwtlQris 
are not s© inipertant, rather standards of a pfiori aehlevimant are 
established and persons and groups of persons are compared to tliis^ 
standards « 

•tndl\^tdual-Ceiitered'tneaaureinent'' Is a fad whtch is Mraly st^rttrig 
and whose Elood ve can escpeet in the not to© distant future* Cencern 
her© Is for* 

(a> The fit of Item reaponies to the raeasurement modal ua^d* 

(b) Items irtiose modtl parameters are invariant from popul^tloia 
to population, 

(c) Person ability or eompetency parafneters of the nsadcl ce 
exhibit iwar lance with respect to the itenu of th^ etfllecttcn 
used to nieasar^ piraens • 

in spite of numerous attempts by others to achlesfc propirclii (b) 
and (c) above only the RaschBlW model, used in the SRlffl sysfcaitij haa 
these properties and the FRM program eitlraatea the model pawiMtiM 
more simply and effiolently of tlie several methodi available to do thla» 
^hus the BHC«FiM systeni beeanes even more important "and unique ai 
measurement prooedures for personnel Reaearch wrk, 

Ijcbiblt 6 Indicates i^htch rRlME Inatrumentallties art likely to be 
useful tools of reaearch tn each of nine research program areas at 
NFKDC. Also apeclfled are those variables which are required tobe 
meaaurad or evaluated and those variables for whteh values ©r atilitl^a 
need to be assessed * 

Let us eKamlne one area In aone ditall, namely program 7^ ^ta^urfe 
Job ptrforwanoe, Sei^attampta at neaauring on-the-job perf©rt»nee hi^re 
never boen vmry suoaessful nor frequent* Indeed this is a dlff ieulE 
research area and useful toala ar^ raodels are rare, Ife do kno^Tp h^v- 
ever* that relevant data are aametinies at hand. For example, itcD^^ill 
(1972) has colleeted "taik'' data frora large samples of persenti^l i« 
ieveral Navy ratings* TTiese data provide '•level of quality'^ respoWia 
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fe^ a^etm ot ^svetal hua<lred tatltig -related tasks and therefore could 
bi used wl^h JAM EWC to cpbt«tti a measure q£ on-the-job conrpetency 
£o« eacti q^epttonn^iw respondent. It appeata that the -'difficulty" ' 
in til is ir^a ciD, Mt l&ast pa*t , be rea&lvad by mn effeetlve Inter- 
r^Sitionihip of th^ iialds tfff Occupaclpnal ^nslyala iiid peycharoetrica « 
Ba«sh field mumm t^ hi\re ai^iadjr developed the appTopriate teelmology 
to mate thie teiy CotitKlbutlo^s to ^olvm the prablem. The crlttcaL pQliiC 
beans to usi the c^teg&rlcal respj^flss data resultiiis itom an Dccupa* 
ttene^l tailt m^twy tegethar ^Ith an aafeasment modtl %Aitch can. accoramo- 
dtw suah data snd subseqiit0tLy derive a mMsurc of person con^etefiey* 
Th^Bft coac«pti ^tLltty bayo^d risear^h at IJPMDC ilaae Task Invantary 

da«a coXlection aa-^ analyae# sti being eatried on elsewhera in the Mav/ 
(c-tf* NCTAI) ind 'by other i#t\^lc€g, 

"Th^ a^sessanat ©f "Ef f#ct ivfettiSB*' of Feraoiis, SyiteM er Objeeta 
BBmmm o£ gmnmm% Wpottance-» "to aie IRira for this probleBa It appears 
th^t m mm Cat<i the .'^bull ttfe hoMi" and Indicate^ for each appll- 
cafti^fij th« isevJie tc be co^§ide^ftd important for assessing effe0ti\fe- 
rtesi as well as the «Lativ# tTrtporcance of eaeh issue , The final aay 
ill tl^ese mattira ls ©f course the perogatlv^e of hi^ le\fel poLicy nakers, 
buftj It th« Miearch l^velj nof^ prcv^lde a procedure mi ftrat 

at«enipt apyroKioiitioni to "#ff ectLveneii'* policy* PRIME instruotantall- 
ti^s tlM amd U%$ B&mm to apP*&ptl«Cely relevant models , enDng others , . 
f^a assessing eff^ctivene is Ve eonalude this aaction by auBaflatl^lng 
aplilLcaelosia in Ejrfiitit^ 6 * 

Ouw p^obleoi Ls caLled '-Asseiament*; nd cur purpoifis v?ere Co 
ds^e1_op a structure fcy ^hicP asi^ssnieiit models could be untqualy 
ch^t^steri^ed and to develop saiessnieiiC modela and their asioctatad computer 
preDgrama ^ic:h CDy^ld be uaeS £ot peaeionnaV reaearcli purpoaes* 

To th«ae e^ds tw aspgtfts ot iai^asment , nanisly meaiuremant and 
ev^^luiatioti were identif lad, Their diaCinctlon lying mainly In the prooesa 
us«d to respond te? tlii itemtf ap^essnent iMtrunent, Thsra ate, 

hcf^iv^sts oitm ^lamemtSj or thelT aoBibinatlonas which can be uaad to 
cLari^fy th« ^3isCiactdo», - 

The parage Eeri cf the ^Ssei^inetit modtLs were identified^ These 
weae va^loiuily CoCTbisiid amd oper^tlonali^td Into either measuranent ox 
ey^ltJiation modeli, Hhm iTtt#ntiom of this proeeas being to make avaluation 
in^at nie^su^enn6n***Llk* by Vai^S developing eval-uatlon ni©deLij lor pplyohot 
onicua reipcnea Ingormaciom, whicli a« pattarttad after lasch latMt^'trait 
ffteastArenjlnt CTOd els . ALso d#^eloped mm mm special oedals for use in 
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PotentUlly Rtli?int PRIME Initrunie-ntalittes to OTMC PiogTaBi Arias 



Assissmtnt Values of 



Potenttal3i 
mim Imtd 



NO. Program Title Variables UtlUtlas yBMPAM :iM 

1. A..,H.^H.. Parformce Ricmt Assi|(meiiti ^ ^ 



Acquisition and 
Initial Semce 

2, Cireer and Occu- Job Parforiami 
pational Dasip Ocwpatlonal Sticeesi 

3, Attitude and Moti- Attitudis, Motiyatlon 
vaticnal ReieaTch Operational Rsadiness 

4, Pfirsonnel EfEac- PersoMil Effactivi- 
tmness mss 

Tiairing rich- Qmml Evaluations 
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6, Test Trainiiig TrainlTig Systiis 
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Keai^reJob Job PeTformame 

Firfornanci 

h HuEin Ferfomaiice 

9. People and Orpn- OrgarlzationaL Effec- 
izational HfinagiMC tivitieis, Motli^itiQn, 

Processss 

10. Manaifflerx SystiBs • Organlzationai Structure 

^ 11. H«t.snMonatto ' Deciito Maldks Methods 



Huian Effectlysncss 



jERJC Processing 



and Values 



iially HelevaiiC PRIME Instrumentalities tc NPRDG Prograni Areas 
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^ . . PRIWE Instrusneatalitles 
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decision loyiklng paradigms and £oy aieess^nt sltuati&iie iot ^hlch an 
Estimation sawpla dees or dota net exigt* 

Analyaii of thmsm m^mln and thmifc associated data is preylded for 
by a iyatem o£ cOTiputar programs called fRIlffi, Iti all| five prograina 
Wire developed 3 illuatrated and dociimantid* Four are tim operating on 
die Ok 1830 cQmpvtar system at OTRDC Ln San BtegOs California, 

Applications of these modeli afid computet prograini to personnel 
reaearcli areas of the NPKDC researcli piograiu was ddaetiased. 
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